{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 保存和序列化模型\n",
    "本指南的第一部分介绍了使用函数式和顺序 API 构建的 Keras 模型的保存和序列化。这两个模型 API 的保存和序列化完全相同。 \n",
    "\n",
    "本指南的第二部分介绍  \"保存和加载分类模型\"，以及子类化 API 与 Keras 顺序和函数式 API 不同。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "\n",
    "tf.keras.backend.clear_session()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第一部分：保存顺序模型或函数式模型\n",
    "让我们考虑以下模型："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"3_layer_mlp\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "digits (InputLayer)          [(None, 784)]             0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 64)                50240     \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 64)                4160      \n",
      "_________________________________________________________________\n",
      "predictions (Dense)          (None, 10)                650       \n",
      "=================================================================\n",
      "Total params: 55,050\n",
      "Trainable params: 55,050\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "from tensorflow import keras\n",
    "from tensorflow.keras import layers\n",
    "\n",
    "inputs = keras.Input(shape=(784,), name='digits')\n",
    "x = layers.Dense(64, activation='relu', name='dense_1')(inputs)\n",
    "x = layers.Dense(64, activation='relu', name='dense_2')(x)\n",
    "outputs = layers.Dense(10, name='predictions')(x)\n",
    "\n",
    "model = keras.Model(inputs=inputs, outputs=outputs, name='3_layer_mlp')\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "（可选）让我们训练该模型，以使其具有要保存的权重值以及优化器状态。\n",
    "\n",
    "当然，你也可以保存从未训练过的模型，但这显然没那么有趣了。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()\n",
    "x_train = x_train.reshape(60000, 784).astype('float32') / 255\n",
    "x_test = x_test.reshape(10000, 784).astype('float32') / 255\n",
    "\n",
    "model.compile(loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
    "              optimizer=keras.optimizers.RMSprop())\n",
    "history = model.fit(x_train, y_train,\n",
    "                    batch_size=64,\n",
    "                    epochs=1)\n",
    "# 在保存之前重置指标，使加载的模型具有相同的状态，因为 model .save_weights 没有保存指标状态\n",
    "model.reset_metrics()\n",
    "\n",
    "# 保存预测以供将来检查\n",
    "predictions = model.predict(x_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 完整的模型保存\n",
    "你可以将使用函数式 API 构建的模型保存到单个文件中。以后，即使你不再有权访问创建模型的代码，也可以从该文件重新创建相同的模型。 该文件包括： \n",
    "1. 模型的架构 \n",
    "2. 模型的权重值\n",
    "3. （在训练过程中获悉）模型的训练配置\n",
    "4. （传递给您的编译参数） 优化器及其状态（如果有的话）（这使您可以在离开的位置重新开始训练）"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 保存模型\n",
    "model.save('path_to_my_model.h5')\n",
    "\n",
    "# 从文件重新创建完全相同的模型\n",
    "new_model = keras.models.load_model('path_to_my_model.h5')\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "# 检查状态是否保留\n",
    "new_predictions = new_model.predict(x_test)\n",
    "np.testing.assert_allclose(predictions, new_predictions, rtol=1e-6, atol=1e-6)\n",
    "\n",
    "# 请注意，优化器状态也保留了：你可以从停止的地方继续培训。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 导出到 SavedModel\n",
    "你可以将整个模型导出为 tensorflow  SavedModel格式。 SavedModel 是 tensorflow 对象的独立序列化格式，受 tensorflow 服务以及除 python 之外的 tensorflow 实现的支持。\n",
    "\n",
    "创建的SavedModel文件包含： \n",
    "1. 一个包含模型权重的TensorFlow检查点。 \n",
    "2. 一个包含基础TensorFlow图的SavedModel原型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 将模型导出为 SavedModel\n",
    "model.save('path_to_saved_model', save_format='tf')\n",
    "\n",
    "# 重新创建完全相同的模型\n",
    "new_model = keras.models.load_model('path_to_saved_model')\n",
    "\n",
    "# 检查状态是否保留\n",
    "new_predictions = new_model.predict(x_test)\n",
    "np.testing.assert_allclose(predictions, new_predictions, rtol=1e-6, atol=1e-6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 只保留结构\n",
    "有时，你只对模型的体系结构感兴趣，而无需保存权重值或优化器。在这种情况下，你可以通过get_config() 方法检索模型的 \"config\"。配置是 python 字典，使你可以重新创建相同的模型——从头开始初始化，而无需先前在训练期间学到的任何信息。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "config = model.get_config()\n",
    "reinitialized_model = keras.Model.from_config(config)\n",
    "\n",
    "new_predictions = reinitialized_model.predict(x_test)\n",
    "assert abs(np.sum(predictions - new_predictions)) > 0.\n",
    "\n",
    "# 您可以使用 from_json() 中的 to_json()，它使用 JSON 字符串存储配置而不是 python 字典。\n",
    "# 这对于将配置保存到磁盘很有用。\n",
    "json_config = model.to_json()\n",
    "reinitialized_model = keras.models.model_from_json(json_config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 只保留权重\n",
    "有时，你只对模型的状态（其权重值）感兴趣，而对体系结构不感兴趣。在这种情况下，你可以通过get_weights() 将权重值检索为 numpy 数组的列表，并可以通过 set_weights 设置模型的状态："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "weights = model.get_weights()  # 检索模型的状态\n",
    "model.set_weights(weights)  # 设置模型的状态"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "您可以结合使用 get_config()/ from_config() 和 get_weights()/ set_weights() 来以相同状态重新创建模型。但是，与 model.save() 不同，它不包括训练配置和优化器。在使用模型进行训练之前，你将不得不再次调用 compile()。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "config = model.get_config()\n",
    "weights = model.get_weights()\n",
    "\n",
    "new_model = keras.Model.from_config(config)\n",
    "new_model.set_weights(weights)\n",
    "\n",
    "new_predictions = new_model.predict(x_test)\n",
    "np.testing.assert_allclose(predictions, new_predictions, rtol=1e-6, atol=1e-6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "get_weight() 和 set_weights(weights) 的保存到磁盘的替代方法是 save_weights(fpath) 和load_weights(fpath)。 下面是保存到磁盘的示例："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#  在磁盘保存为 json 格式\n",
    "json_config = model.to_json()\n",
    "with open('model_config.json', 'w') as json_file:\n",
    "    json_file.write(json_config)\n",
    "# 将权重保存在磁盘\n",
    "model.save_weights('path_to_my_weights.h5')\n",
    "\n",
    "# 从保存的两个文件中加载模型\n",
    "with open('model_config.json') as json_file:\n",
    "    json_config = json_file.read()\n",
    "new_model = keras.models.model_from_json(json_config)\n",
    "new_model.load_weights('path_to_my_weights.h5')\n",
    "\n",
    "new_predictions = new_model.predict(x_test)\n",
    "np.testing.assert_allclose(predictions, new_predictions, rtol=1e-6, atol=1e-6)\n",
    "\n",
    "# 注意：最简单的建议方法就是这样\n",
    "model.save('path_to_my_model.h5')\n",
    "del model\n",
    "model = keras.models.load_model('path_to_my_model.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 使用 tensorflow 检查点保存权重\n",
    "注意：save_weight s可以创建 Keras HDF5 格式或 tensorflow Checkpoint 格式的文件。\n",
    "\n",
    "该格式是从你提供的文件扩展名中推断出来的：如果它是 \" .h5\" 或 \".keras\"，则框架使用 Keras HDF5格式。其他所有内容默认为Checkpoint。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save_weights('path_to_my_tf_checkpoint')\n",
    "\n",
    "# 为了明确，可以通过 save_format 参数显式传递格式，该参数的值可以为 \"tf\" 或 \"h5\"：\n",
    "model.save_weights('path_to_my_tf_checkpoint', save_format='tf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 第二部分：保存和加载子类模型\n",
    "顺序模型和函数式模型是表示层 DAG 的数据结构。这样，它们可以安全地序列化和反序列化。 \n",
    "\n",
    "子类化模型的不同之处在于，它不是数据结构，而是一段代码。通过调用方法的主体定义模型的体系结构，这意味着无法安全地序列化模型的体系结构。要加载模型，你需要有权访问创建模型的代码（模型子类的代码）。或者，你可以将此代码序列化为字节码（例如，通过 pickling），但这是不安全的，并且通常不具备可移植性。\n",
    "\n",
    "有关这些差异的更多信息，请参阅文章 [What are Symbolic and Imperative APIs in TensorFlow 2.0?](https://medium.com/tensorflow/what-are-symbolic-and-imperative-apis-in-tensorflow-2-0-dfccecb01021)。 \n",
    "\n",
    "让我们考虑以下子类化模型，该子类化模型与第一部分中的模型具有相同的结构："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class ThreeLayerMLP(keras.Model):\n",
    "\n",
    "  def __init__(self, name=None):\n",
    "    super(ThreeLayerMLP, self).__init__(name=name)\n",
    "    self.dense_1 = layers.Dense(64, activation='relu', name='dense_1')\n",
    "    self.dense_2 = layers.Dense(64, activation='relu', name='dense_2')\n",
    "    self.pred_layer = layers.Dense(10, name='predictions')\n",
    "\n",
    "  def call(self, inputs):\n",
    "    x = self.dense_1(inputs)\n",
    "    x = self.dense_2(x)\n",
    "    return self.pred_layer(x)\n",
    "\n",
    "def get_model():\n",
    "  return ThreeLayerMLP(name='3_layer_mlp')\n",
    "\n",
    "model = get_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "首先，无法保存从未使用过的子类模型。 这是因为需要在某些数据上调用子类模型才能创建其权重。 在调用模型之前，它不知道应期望的输入数据的形状和 dtype ，因此无法创建其权重变量。你\n",
    "\n",
    "可能还记得在第一部分的函数式模型中，输入的形状和dtype是预先指定的（通过keras.Input(...)）——这就是为什么函数式模型在输入后立即具有状态的原因实例化。 让我们训练模型，使其具有状态："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()\n",
    "x_train = x_train.reshape(60000, 784).astype('float32') / 255\n",
    "x_test = x_test.reshape(10000, 784).astype('float32') / 255\n",
    "\n",
    "model.compile(loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
    "              optimizer=keras.optimizers.RMSprop())\n",
    "history = model.fit(x_train, y_train,\n",
    "                    batch_size=64,\n",
    "                    epochs=1)\n",
    "# 在保存之前重置度量，以便加载的模型具有相同的状态，因为度量状态不被 Model.save_weights 保留\n",
    "model.reset_metrics()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "有三种不同的方法来保存和恢复子类模型。以下各节提供了有关这三种方法的更多详细信息：\n",
    "### 方法1\n",
    "保存子类模型的推荐方法是使用 save_weights 创建一个 tensorflow  SavedModel检查点，该检查点将包含与该模型关联的所有变量的值：\n",
    "- 图层的权重\n",
    "- 优化器状态\n",
    "- 与状态模型指标相关的任何变量"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.save_weights('path_to_my_weights', save_format='tf')\n",
    "\n",
    "# 保存预测值以供将来检查\n",
    "predictions = model.predict(x_test)\n",
    "# 保存第一批的损失值，以便稍后断言优化器状态已被保留\n",
    "first_batch_loss = model.train_on_batch(x_train[:64], y_train[:64])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "要还原模型，你需要访问创建模型对象的代码。 \n",
    "\n",
    "注意：为了恢复优化器状态和任何有状态度量的状态，你应该编译模型（使用与之前完全相同的参数），并在调用 load_weights 之前对某些数据进行调用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 重建模型\n",
    "new_model = get_model()\n",
    "new_model.compile(loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
    "                  optimizer=keras.optimizers.RMSprop())\n",
    "\n",
    "# 初始化优化器使用的变量以及任何有状态的度量变量\n",
    "new_model.train_on_batch(x_train[:1], y_train[:1])\n",
    "\n",
    "# 加载旧模型的状态\n",
    "new_model.load_weights('path_to_my_weights')\n",
    "\n",
    "# 检查模型状态是否已保留\n",
    "new_predictions = new_model.predict(x_test)\n",
    "np.testing.assert_allclose(predictions, new_predictions, rtol=1e-6, atol=1e-6)\n",
    "\n",
    "# 优化器的状态也会被保留，因此可以在中断的地方继续训练\n",
    "new_first_batch_loss=new_model.train_on_batch(x_train[:64], y_train[:64])\n",
    "assert first_batch_loss == new_first_batch_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 方法2\n",
    "第二种方法是使用 model.save 保存整个模型，并使用 load_model 恢复以前存储的子类模型。以下代码段描述了如何实现它们："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 保存模型\n",
    "model.save('path_to_my_model',save_format='tf')\n",
    "\n",
    "# 从文件重新创建完全相同的模型\n",
    "new_model = keras.models.load_model('path_to_my_model')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 方法3\n",
    "第三种方法是使用 tf.saved_model.save。这等效于 model.save 中的 tf 格式。你可以再次调用load_model 以恢复先前保存的子类模型。以下代码段描述了如何实现它们。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 保存模型\n",
    "tf.saved_model.save(model,'my_saved_model')\n",
    "# 恢复模型\n",
    "restored_saved_model = keras.models.load_model('my_saved_model')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6rc1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
